IN THE SPECIFICATION: 

Please amend paragraphs [004], [006], [008], [009], [Oil], [012], [016], [053], [098], [099], 
[103], [108], [110], [111], [139], [144], [146], [150], [153], [156], [157], [159], [164]-[168], [171] 
and [173], as shown below, in which deleted terms are shown with strikethrough and/or added terms 
are shown with underscoring. 

Paragraph [004] 

An example of such devices for monitoring around a vehicle which extracts an object, 
which may collide with the vehicle, from an image around the vehicle captured by an infrared 
camera is as follows. That is, in the device, the captured infrared image is subjected to a binary 
(2-level) thresholding process, and an area to which bright (white) portions are concentrated is 
searched for. Then, it is determined whether the area is a head portion of a pedestrian by using 
the aspect ratio (ratio of length to width) and the sufficiency rate of the area, and further 
calculating the distance between the vehicle and the area using the actual surface area and the 
position of the center of gravity in the image. If the area of the head portion of a pedestrian is 
determined, an area which forms the body of the pedestrian is determined by calculating the 
height of the pedestrian in the image based on the distance between the area determined to be the 
head portion and the camera, and an average height of an adult. These areas are displayed to be 
distinguished from the other regions of the image. In this manner, the position of the entire body 
of the pedestrian in the infrared image is determined, and this information is displayed for the 
driver so as to effectively assist the vision of the driver (refer to Japanese Unexamined Patent 
Application, First Publication No. Hei 11-328364, for example). 

Paragraph [006] 

Accordingly, since the barycentric coordinates of targeted objects, such as pedestrians, in 
the picture cannot be fixed with respect to the distance. Therefore, it is not possible to extract the 
targeted objects stably such as pedestrians which may collide with the vehicle, if the extraction is 
carried out based on the shape of at least the pedestrians* height, head, body as in the above- 
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mentioned conventional device. 



Paragraph [008] 

In order to achieve the above objects, the present invention provides a device for 
monitoring around a vehicle capable of detecting objects present around the vehicle based on an 
image captured by at least one infrared camera member provided with said vehicle such that said 
device comprises a pedestrian's head area calculating unit (for example, Steps S91 to S94 in the 
embodiment of the present invention) which establishes an area which is supposed to correspond 
to a head of the pedestrian as a reference area, a pedestrian's over-shoulder area calculating unit 
(for example, Step S95 in the embodiment of the present invention) which establishes two object 
areas which are supposed to correspond to over-shoulder areas of the pedestrian on both sides of 
the reference area, and a pedestrian's shape acknowledging unit (for example, Steps S96, S98, 
and SI 00 in the embodiment of the present invention) which acknowledging acknowledges the 
pedestrian who is in the captured image according to a feature in a luminance in the reference 
areas and a feature in a luminance in the object areas. 

Paragraph [009] 

According to the device for monitoring around a vehicle described above, the device for 
monitoring around a vehicle establishes an area which is supposed to correspond to a head of a 
pedestrian as a reference area by a pedestrian's head area calculating unit due to a theory that the 
head of the pedestrian has a high luminance in an image which is captured by an infrared ray 
camera. Next, two object areas which are supposed to correspond to over-shoulder areas of the 
pedestrian are established on both sides of the reference area by the pedestrian's over-shoulder 
area calculating unit. By doing this, the pedestrian's shape acknowledging unit can 
acknowledging acknowledge the area which is supposed to correspond to the head and the 
shoulders of the pedestrian who is in the image which is captured by an the infra-red ray camera 
due to a theory that a the head of the pedestrian has a high luminance contrast with reference to a 
luminance contrast besides the head (background) in upward spaces en over the shoulders 
according to a feature in a luminance in the reference areas and a feature in a luminance in the 



4 



object areas. 
Paragraph [Oil] 

In the device for monitoring around a vehicle which is provided with above structure, it 
is possible to prevent the an image which belongs to an area which is different from the targeted 
area from entering the targeted area because the of object incline in the infra-red ray image. 

Paragraph [012] 

A device for monitoring around a vehicle further comprises a pedestrian's shoulder area 
calculating unit (for example, Steps S97 and S99 in the embodiment of the present invention) 
which establishes another object areas for acknowledging the pedestrian's arms downwardly to 
the object areas^ respectively^ such that the pedestrian's shape acknowledging unit acknowledges 
the pedestrian in the captured image according to a feature in a luminance in the object areas and 
the other object areas. 

Paragraph [016] 

FIG. 2 is a perspective diagram showing positions of an infrared ray camera, a sensor, a 
display, etc., attached to a vehicle according to the embodiment of the present invention; 

Paragraph [053] 

After the center of gravity, the surface area, and the aspect ratio of the circumscribed 
square are calculated, and a process for tracking the object in relation to time, i.e., a process in 
which the same object is recognized every sampling period, is carried out (step S9). In the 
tracking process in relation to time, objects A and B are extracted at time k, which is obtained by 
discretization of analog time t using the sampling period, for instance, and it is determined that if 
objects C and D, which are extracted at time (k+1), are the same objects as the objects A and B, 
respectively. Then, if it is determined that the objects A and B and the objects C and D are the 
same objects, labels of the objects C and D are changed to label A and B, respectively, to carry 
out the tracking process in relation to time. 
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Paragraph [098] 

Also, in step S46, if the height position Yt of the upper end of the object from the surface 
of the road is less than the threshold value TH5 (i.e., "YES" in step S46), it is determined 
whether or not the luminance dispersion Var_A3 of the mask area AREA3 is greater than the 
threshold value TH6 (in step S47). This process will be explained with reference to FIGS. 16A 
through 16C which show the luminance dispersion in the mask area ARE A3 for the case where 
the object is a part or the a whole of a pedestrian, or a wall. 

Paragraph [099] 

As shown in FIG. 16A, when only a head portion of a pedestrian is extracted by the binary 
process using the width of the mask area ARE A3 as the width of the binary object, difference in 
the degree of luminance is caused between the head portion and the lower half of the body 
portion. Also, as shown in FIG. 1 6B, when the entire body or at least the upper half of the body 
of a pedestrian is extracted using the binary thresholding method, difference in the degree of 
luminance is generated between the body portion of the pedestrian and the background. On the 
other hand, as shown in FIG. 16C, for the object in which the difference in the temperature is 
small overall over all the parts of the objects, such as a wall, difference in the degree of 
luminance is also small between portions extracted by the binary process and portions not 
extracted by the binary process. Moreover, the object is formed by portions of straight lines as in 
the ARE A3. For this reason, the luminance dispersion Var_A3 in the ARE A3 is high for a 
pedestrian, and low for an object, such as a wall. 

Paragraph [103] 

Also, in step S48, if the difference between the maximum value Max_Rate and the 
minimum value Min_Rate of the Rate, which indicates the ratio of the surface area between the 
circumscribed square and the binary object within a certain period of time, is less than the 
threshold value TH7 (i.e., "YES" in step S48), in the next step, it is determined whether or not 
there exists an area which is supposed to correspond to be a head of the pedestrian in an object 
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which is captured in an area AREAO (Step S48-1). 
Paragraph [108] 

Also, if an area which is inferred to correspond to the shoulders and the arms of the 
pedestrian exists in the object which is captured in the area AREAO in the step S48-2 (i.e., "YES" 
in the step S48-2), in the next step, the image processing unit 1 then carries out the pedestrian 
determination process for the shape of each extracted object in further detail. 

Paragraph [110] 

In step S50, if the height position Yt of the upper end of the object from the surface of the 
road is equal to or less than the threshold value TH8 (i.e., tc NO" in step S£± S50), the process 
proceeds to step S51 shown in the flowchart in FIG. 9. In step S51, it is determined if the width 
AWb of the binary object is equal to or less than the threshold value TH9, which is a value 
appropriate for the width of the body of a pedestrian) to determine if it is a lower half of the body 
of a pedestrian or a sitting pedestrian. 

Paragraph [111] 

FIG. 9 is a flowchart showing the procedure for distinguishing a pedestrian whose lower 
half of the body is extracted by the binary process or who is sitting. In step S51 of the flowchart, 
if the width AWb of the binary object is equal to or less than the threshold value TH9 (i.e., 
"YES" in step S51), it is determined whether or not the height AHg of the gray scale object is 
less than the threshold value TH10, which is a value appropriate for the height of a pedestrian, in 
order to determine if the object is a sitting pedestrian (in step S52). Optionally, after step S51, it 
may be determined whether it is raining around the vehicle. If it is raining, the process proceeds 
to step S54, while if it is not raining, the process proceeds to step S52. 

Paragraph [139] 

FIG. 13 is a flowchart showing a procedure for the case where the object is a plurality of 
pedestrians walking in parallel. If it is determined that the value of RATE, which is the ratio of 
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the surface area of the circumscribed square to that of the binary image within a certain period of 
time, is less than the threshold value TH22 in step S76 (i.e., "YES" in step S76), it is determined 
that whether the value of Asp, which expresses the aspect ratio of the circumscribed square for 
the binary object, is equal to or greater than the threshold value TH23 and is equal to or less than 
the threshold value TH14, i.e., the value appropriate for determining pedestrians walking in 
parallel (in step S77). 

Paragraph [144] 

In FIG. 17, it is detected whether or not there exists a head of the pedestrian by utilizing a 
feature in the luminance in each area in the image such that the pedestrian's head has a higher a 
luminance contrast than the luminance contrast in a background on both sides of the head. That 
is, as shown in FIGS. 18A and 18B, a projection area AREA4 is established (Step S91) in an area 
which is indicated "xh [m]" as a real space in an upper portion of an object area AREA 0 which 
is extracted as a gray scale image. After that, a luminance projection (a distribution of an 
integrated luminance in a horizontal direction which is formed by integrating the luminance in 
each pixel in a vertical direction) is calculated in a vertical direction (step S92) so as to detect a 
horizontal direction coordinate xCC which indicates a maximum peak with reference to an upper 
left reference point O (step S93). 

Paragraph [146] 

As shown in FIG. 19A, a reference area mask Mask_C is established (step S94) in an area 
which is inferred to correspond to a head position of the pedestrian with reference to the 
horizontal direction position xCC which is detected in the step S93. Simultaneously, an object 
area masks MaskJLT and Mask_RT for the object area 1 are established (step S95) which are 
inferred to correspond to a space above the shoulders in both sides of the pedestrian's head. 

Paragraph [150] 

In the step S96, if the pedestrian's head does not exist (i.e., "NO" in the step S96), the 
process goes to a step S49 shown in FIG. 8 because it is an identical to a case for "NO" in the step 
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S48-1 shown in FIG. 8. Thus, it is determined that the object which is captured in the area 
AREAO is not a pedestrian (step S49); thus, the pedestrian determination pric e ss process is 
completed. 

Paragraph [153] 

Next, it is determined whether or not an area exists which indicates the shoulders and 
arms of the pedestrian by taking features of the luminance in each area (each section) in the 
image into account that the area which indicates the shoulders and arms of the pedestrian exists 
so as to be distant from the head by approximately the same distance from the position of the 
head and the luminance contrast of the shoulders and arms of the pedestrian is different from the 
luminance contrast in a background on both sides of the head which is positioned en above the 
shoulders of the pedestrian. In this manner, it is possible to further improve an accuracy for 
detecting the pedestrian. More specifically, as shown in FIG. 19 A, other object area masks 
Mask_LU and Mask_RU as the object area 2 are establised (step S97) which correspond to the 
shoulders and arms under the object area masks Mask_LT and Mask_RT which are disposed in 
on both sides of the pedestrian's head which are used for determining a head in the step S96. 

Paragraph [156] 

Also, in the step S98, if the area which corresponds to the shoulders and arms of the 
pedestrian exists (i.e., "YES" in the step S98), the process goes to the next step S99. 

Paragraph [157] 

Here, in the step S98, it should be noted that a relativity error value between the object 
area masks Mask_LT and the the other object area masks MaskJAJ is indicated by an ErrorJL so 
as to specifically compare the feature of the luminance in each area (each section) on the image. 
Also, an relativity error value between the object area masks Mask_RT and the the other object 
area masks Mask_RU is indicated by an Errorjl. Consequently, the threshold values TH26 
which are indicated below and the above relativity error are compared. 
"Error_R">TH26-(13) 
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, Error_L n >TH26-(14) 



Paragraph [159] 

Next, as shown in FIG. 20, two areas such as Mask_L and Mask_R are established as an 
object area 3 (step S99) therebeneath on both sides of the targeted object area AREAO which is 
extracted as a gray scale image. Disparities for these areas (actual spatial distances) are 
calculated according to a stereo images which are disposed horizontally. Here, it should be noted 
that projected size for a W2_Space and a MaskJWl indicate a designated size in the actual space. 

Paragraph [164] 

As explained above, in the device for monitoring around a vehicle according to the 
embodiment of the present invention, an objects such as a pedestrian is extracted from the gray 
scale image which is captured by an infra-red ray camera by performing a binary process. After 
that, an area which is inferred to correspond to the pedestrian's head is establised as a reference 
area on the gray scale image which contains the binary object by the pedestrian's head area 
calculating unit. Consequently, the a pedestrian's over-shoulder area calculating unit establishes 
two object areas which are inferred to correspond to spaces above the shoulders of the pedestrian 
on both sides of the reference area upwardly. 

Paragraph [165] 

In this manner, the pedestrian's shape acknowledging unit acknowledges an area which 
corresponds to the head and shoulders of the pedestrian in an image which is captured by an 
infra-red ray camera by taking a feature into account that the pedestrian's head has a higher 
luminance contrast than the luminance contrast in a background which is on both sides of the 
head above the shoulders of the pedestrian. 

Paragraph [166] 
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Furthermore, an object area which is inferred to correspond to an area above the 
shoulders of the pedestrian and another object area which is established by the pedestrian's 
shoulder area calculating unit so as to be inferred to correspond to the shoulders and arms of the 
pedestrian are compared. Thus, the area which corresponds to the shoulders and arms of the 
pedestrian in an image which is captured by an infra-red ray camera is acknowledged according 
to a theory that the luminance contrast in the shoulders and arms of the pedestrian is different 
from the luminance contrast in a background in on both sides of the pedestrian's head above the 
shoulders. 

Paragraph [167] 

Accordingly, it is understood that a distance for an entire pedestrian is equal to a distance 
to the shoulders and arms. Therefore, under the condition that the distance for the entire 
pedestrian and the distance to the shoulders and arms are compared, if a difference between the 
above distances is less than a certain value, the pedestrian's shape acknowledging unit can 
acknowledge that the binary object which contains the area which corresponds to the head and 
shoulders of the pedestrian and the area which corresponds to the shoulders and arms indicate a 
pedestrian. 

Paragraph [168] 

Accordingly, if the pedestrian is captured in an image which is captured by an infra-red 
ray camera in an inclining manner, an area which corresponds to the head and the shoulders of 
the pedestrian in which an outstanding feature of the pedestrian may be observed is detected from 
the image as the object by using the two object areas which are disposed above the reference area 
such that an area which is not an targeted area should not enter in the object area. Next, an area 
which corresponds to the shoulders and arms in which an outstanding feature of the pedestrian 
may be observed is detected. Furthermore, it is confirmed that a distance from an area which 
contains an entire pedestrian is equal to a distance from an area which corresponds to the 
shoulders and arms of the pedestrian. After that, it is acknowledged that the binary object which 
contains an area which corresponds to the pedestrian's head and the shoulders and an area which 



11 



corresponds to the shoulders and arms indicates the pedestrian. In this manner, it is possible to 
realize an effect in which it is possible to improve an accuracy for detecting the pedestrian. 

Paragraph [171] 

According to the device for monitoring around a vehicle in another aspect of the present 
invention, it is possible to prevent an image in an area which is different from the intended area 
from entering the object area by inclining the object in an image which is captured by an infra- 
red ray camera. 

Paragraph [173] 

According to the device for monitoring around a vehicle in still an other aspect of the 
present invention, an object area which is inferred to correspond to the space above the 
pedestrian's shoulders and another object area which is inferred to correspond to the pedestrian's 
shoulders and arms which are established by the pedestrian's shoulder area calculating unit are 
compared with each other. Consequently, the pedestrian's shape acknowledging unit can 
acknowledge an area which corresponds to the pedestrian's shoulder and arms in an image which 
are is captured by an infra-red ray camera by utilizing a feature that the pedestrian's shoulders and 
arms have a different luminance contrast from the luminance contrast in a background on both 
side of the head. 
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